129 research outputs found
Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids
Real-life control tasks involve matters of various substances---rigid or soft
bodies, liquid, gas---each with distinct physical behaviors. This poses
challenges to traditional rigid-body physics engines. Particle-based simulators
have been developed to model the dynamics of these complex scenes; however,
relying on approximation techniques, their simulation often deviates from
real-world physics, especially in the long term. In this paper, we propose to
learn a particle-based simulator for complex control tasks. Combining learning
with particle-based systems brings in two major benefits: first, the learned
simulator, just like other particle-based systems, acts widely on objects of
different materials; second, the particle-based representation poses strong
inductive bias for learning: particles of the same type have the same dynamics
within. This enables the model to quickly adapt to new environments of unknown
dynamics within a few observations. We demonstrate robots achieving complex
manipulation tasks using the learned simulator, such as manipulating fluids and
deformable foam, with experiments both in simulation and in the real world. Our
study helps lay the foundation for robot learning of dynamic scenes with
particle-based representations.Comment: Accepted to ICLR 2019. Project Page: http://dpi.csail.mit.edu Video:
https://www.youtube.com/watch?v=FrPpP7aW3L
Propagation Networks for Model-Based Control Under Partial Observation
There has been an increasing interest in learning dynamics simulators for
model-based control. Compared with off-the-shelf physics engines, a learnable
simulator can quickly adapt to unseen objects, scenes, and tasks. However,
existing models like interaction networks only work for fully observable
systems; they also only consider pairwise interactions within a single time
step, both restricting their use in practical systems. We introduce Propagation
Networks (PropNet), a differentiable, learnable dynamics model that handles
partially observable scenarios and enables instantaneous propagation of signals
beyond pairwise interactions. Experiments show that our propagation networks
not only outperform current learnable physics engines in forward simulation,
but also achieve superior performance on various control tasks. Compared with
existing model-free deep reinforcement learning algorithms, model-based control
with propagation networks is more accurate, efficient, and generalizable to
new, partially observable scenes and tasks.Comment: Accepted to ICRA 2019. Project Page: http://propnet.csail.mit.edu
Video: https://youtu.be/ZAxHXegkz4
Novel design and simulation of a hybrid solar electricity system with organic Rankine cycle and PV cells
The proposed system mainly consists of flat-plate compound parabolic concentrators (CPCs) integrated with photovoltaic (PV) cells and organic Rankine cycle (ORC). The technologies of CPC, PV cell and ORC are analyzed, and feasibility of the hybrid solar electricity system is demonstrated. Novel configuration for the hybrid electricity generation is carefully designed to react to different operating conditions. Fundamentals of the innovative system are illustrated, and mathematical models are developed to study the heat transfer and energy conversion processes. The results indicate that the lowtemperature solar thermal power generation integrated PV cells can produce much more electric per unit surface area than side-by-side PV panels and CPC-ORC modules. © The Author 2010. Published by Oxford University Press. All rights reserved
Dynamic-Resolution Model Learning for Object Pile Manipulation
Dynamics models learned from visual observations have shown to be effective
in various robotic manipulation tasks. One of the key questions for learning
such dynamics models is what scene representation to use. Prior works typically
assume representation at a fixed dimension or resolution, which may be
inefficient for simple tasks and ineffective for more complicated tasks. In
this work, we investigate how to learn dynamic and adaptive representations at
different levels of abstraction to achieve the optimal trade-off between
efficiency and effectiveness. Specifically, we construct dynamic-resolution
particle representations of the environment and learn a unified dynamics model
using graph neural networks (GNNs) that allows continuous selection of the
abstraction level. During test time, the agent can adaptively determine the
optimal resolution at each model-predictive control (MPC) step. We evaluate our
method in object pile manipulation, a task we commonly encounter in cooking,
agriculture, manufacturing, and pharmaceutical applications. Through
comprehensive evaluations both in the simulation and the real world, we show
that our method achieves significantly better performance than state-of-the-art
fixed-resolution baselines at the gathering, sorting, and redistribution of
granular object piles made with various instances like coffee beans, almonds,
corn, etc.Comment: Accepted to Robotics: Science and Systems (RSS 2023). The first two
authors contributed equally. Project Page:
https://https://robopil.github.io/dyn-res-pile-mani
Causal Discovery in Physical Systems from Videos
Causal discovery is at the core of human cognition. It enables us to reason about the environment and make counterfactual predictions about unseen scenarios that can vastly differ from our previous experiences. We consider the task of causal discovery from videos in an end-to-end fashion without supervision on the ground-truth graph structure. In particular, our goal is to discover the structural dependencies among environmental and object variables: inferring the type and strength of interactions that have a causal effect on the behavior of the dynamical system. Our model consists of (a) a perception module that extracts a semantically meaningful and temporally consistent keypoint representation from images, (b) an inference module for determining the graph distribution induced by the detected keypoints, and (c) a dynamics module that can predict the future by conditioning on the inferred graph. We assume access to different configurations and environmental conditions, i.e., data from unknown interventions on the underlying system; thus, we can hope to discover the correct underlying causal graph without explicit interventions. We evaluate our method in a planar multi-body interaction environment and scenarios involving fabrics of different shapes like shirts and pants. Experiments demonstrate that our model can correctly identify the interactions from a short sequence of images and make long-term future predictions. The causal structure assumed by the model also allows it to make counterfactual predictions and extrapolate to systems of unseen interaction graphs or graphs of various sizes
Causal Discovery in Physical Systems from Videos
Causal discovery is at the core of human cognition. It enables us to reason
about the environment and make counterfactual predictions about unseen
scenarios that can vastly differ from our previous experiences. We consider the
task of causal discovery from videos in an end-to-end fashion without
supervision on the ground-truth graph structure. In particular, our goal is to
discover the structural dependencies among environmental and object variables:
inferring the type and strength of interactions that have a causal effect on
the behavior of the dynamical system. Our model consists of (a) a perception
module that extracts a semantically meaningful and temporally consistent
keypoint representation from images, (b) an inference module for determining
the graph distribution induced by the detected keypoints, and (c) a dynamics
module that can predict the future by conditioning on the inferred graph. We
assume access to different configurations and environmental conditions, i.e.,
data from unknown interventions on the underlying system; thus, we can hope to
discover the correct underlying causal graph without explicit interventions. We
evaluate our method in a planar multi-body interaction environment and
scenarios involving fabrics of different shapes like shirts and pants.
Experiments demonstrate that our model can correctly identify the interactions
from a short sequence of images and make long-term future predictions. The
causal structure assumed by the model also allows it to make counterfactual
predictions and extrapolate to systems of unseen interaction graphs or graphs
of various sizes.Comment: NeurIPS 2020. Project page: https://yunzhuli.github.io/V-CDN
RoboCook: Long-Horizon Elasto-Plastic Object Manipulation with Diverse Tools
Humans excel in complex long-horizon soft body manipulation tasks via
flexible tool use: bread baking requires a knife to slice the dough and a
rolling pin to flatten it. Often regarded as a hallmark of human cognition,
tool use in autonomous robots remains limited due to challenges in
understanding tool-object interactions. Here we develop an intelligent robotic
system, RoboCook, which perceives, models, and manipulates elasto-plastic
objects with various tools. RoboCook uses point cloud scene representations,
models tool-object interactions with Graph Neural Networks (GNNs), and combines
tool classification with self-supervised policy learning to devise manipulation
plans. We demonstrate that from just 20 minutes of real-world interaction data
per tool, a general-purpose robot arm can learn complex long-horizon soft
object manipulation tasks, such as making dumplings and alphabet letter
cookies. Extensive evaluations show that RoboCook substantially outperforms
state-of-the-art approaches, exhibits robustness against severe external
disturbances, and demonstrates adaptability to different materials.Comment: Project page: https://hshi74.github.io/robocook
- …